A Symbolic Descriptive System for Facial Expression Conveying Linguistic Information In Signing

The kinds of change in facial expression which can be produced by a sender's neural control of facial muscles, and, at the same time, can be visually perceived by a listener are described in discrete symbols. The linguistic information supplemented by the facial expressions in signing is categorized into syllabic, lexical and syntactic components. In order to control the facial expressions for this linguistic information, the timing and duration of the symbols are specified, and, by applying a set of concatenation rules, the control commands are generated. The adequacy of the symbolic descriptive system and its control rules have been examined through a computer synthesis of artificial facial expression, taking the example of transmitting syntactic information in traditional Japanese sign language. These facial expressions are combined with the symbolic descriptive system for hand shapes and arm actions and their control rules for the pictorial display of signing gestures. 1. SYMBOLIC DESCRIPTION 1. 1 . Elements of Facial Expression The kinds of change in facial expression which can be produced by a sender's neural control of facial muscles [References 1 and 2], and, at the same time, can be perceived visually by a listener, are described in discrete symbols. Parameters used for the element of facial expression are as follows: 1. Opening and closing of the eyelids 2. the eye direction 3. raising, knitting and lowering of the eyebrows 4. opening of the nostrils 5. opening or closing of the mouth 6. protrusion or corner pull of the lips 7. popping up or drawing of the chin and 8. nod or shake of the head The correspondence between the parameters of the elements and their symbols in a symbolic ELEMENTS SYMBOLS PARAMETERS EYELIDS: NEUTRAL EL0 WIDELY OPENED EL+ NARROWED, OR ELCLOSED EL-EYE DIRECTION: NEUTRAL EY0 DIRECTLY UP EYu+, u++ DOWN EYd+, d++ LEFT, OR EYl+, l++ RIGHT EYr+, r++ EACH IN TWO STEPS DIRECT TO A PARTICULAR POINT EYxy EYEBROWS: NEUTRAL EB0 THE INSIDES RAISED, OR EBi+, i++ KNITTED EBi-, i-THE LATERAL SIDES RAISED, OR EBl+, l++ LOWERED EBl-, l-EACH IN TWO STEPS. NOSTRILS: NEUTRAL NR0 WIDELY OPENED NR+ MOUTH: NEUTRAL MO0 WIDELY OPENED MO+ NARROWED, OR MOCLOSED MO-THE LIPS PROTRUDED, OR LP+, ++ THE CORNERS PULLED LATERALLY LP-, -EACH IN TWO STEPS CHIN: NEUTRAL CN0 POPPING UP, OR CN+, ++ DRAWING IN THE CHIN CN+, ++ EACH IN TWO STEPS HEAD: NEUTRAL HD0 NOD, OR HDN SHAKE. HDS TILT FORWARD, OR HD+, ++ BACKWARD HD-, -EACH IN TWO STEPS Tabl e 1 : Correspondence between the parameters of elements and their symbols in a symbolic descriptive system of the facial expressions. ! " # $ ISCA Archive %&&'''( ( & )